Subjective Assessment Experiments That Recruit Few Observers With Repetitions (FOWR)

Recent studies have shown that it is possible to characterize subject bias and variance in subjective assessment tests. Apparent differences among subjects can, for the most part, be explained by random factors. Building on theory, we propose a test design where four six team members each rate stimuli multiple times. The results are comparable high performing objective metric. This provides quick simple way analyze new technologies perform pre-tests assessment.